Robust and Reliable Feature Extractor Training by Using Unsupervised Pre-training with Self-Organization Map
نویسندگان
چکیده
Recent research has shown that deep neural network is very powerful for object recognition task. However, training the deep neural network with more than two hidden layers is not easy even now because of regularization problem. To overcome such a regularization problem, some techniques like dropout and de-noising were developed. The philosophy behind de-noising is to extract more robust features from the training data. For that purpose, randomly corrupted input data are used for training an auto-encoder or Restricted Boltzmann machine (RBM). In this paper, we propose unsupervised pre-training with a SelfOrganization Map (SOM) to increase robustness and reliability of feature extraction. The basic idea is that instead of random corruption, our proposed algorithm works as a feature extractor so that corrupted input maintains the main skeleton or structure of original data. As a result, our proposed algorithm can extract more robust features related to input data.
منابع مشابه
Distortion tolerant pattern recognition based on self-organizing feature extraction
A generic, modular, neural network-based feature extraction and pattern classification system is proposed for finding essentially two-dimensional objects or object parts from digital images in a distortion tolerant manner, The distortion tolerance is built up gradually by successive blocks in a pipeline architecture. The system consists of only feedforward neural networks, allowing efficient pa...
متن کاملSelf-organization for the Detection of Local Features
A massive advancement in image processing technologies supports a rapid on-line classification in well-defined pattern recognition tasks. Yet in general, there remains an urgent need for robust pre-processing techniques applicable independent of the task to be solved. Currently, two such approaches based on artificial neural networks are known from the literature, namely the so-called Extreme L...
متن کاملOptimal Tailoring of Trajectories, Growing Training Sets and Recurrent Networks for Spoken Word Recognition
A novel system that efficiently integrates two types of neural networks for reliably performing isolated word recognition is described. The recognition system comprises of a feature extractor that includes a Self Organizing Map for an optimal tailoring of trajectory representations of words in reduced dimension feature spaces. Experimental results indicate that such lower dimensional trajectori...
متن کاملTransfer Learning by Supervised Pre-training for Audio-based Music Classification
Very few large-scale music research datasets are publicly available. There is an increasing need for such datasets, because the shift from physical to digital distribution in the music industry has given the listener access to a large body of music, which needs to be cataloged efficiently and be easily browsable. Additionally, deep learning and feature learning techniques are becoming increasin...
متن کاملA scalable approach to using DNN-derived features in GMM-HMM based acoustic modeling for LVCSR
We present a new scalable approach to using deep neural network (DNN) derived features in Gaussian mixture density hidden Markov model (GMM-HMM) based acoustic modeling for large vocabulary continuous speech recognition (LVCSR). The DNN-based feature extractor is trained from a subset of training data to mitigate the scalability issue of DNN training, while GMM-HMMs are trained by using state-o...
متن کامل